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ABSTTIACT 


The  results  from  a  test  performed  to  validate  the  Navy  Atmospheric  Boundaiy  Layer 
(NABL)  model  are  presented.  Average  statistics  indicate  that  NABL  forecast  temperature  profiles 
compare  favorably  with  those  from  a  40  km  horizontal  resolution  version  of  the  Navy 
Operational  Regional  Atmospheric  Prediction  System  (NORAPS),  but  that  negative  (dry)  moisture 
biases  are  more  severe  in  the  NABL  model.  Test  results  indicate  that  the  NABL  model  is 
superior  to  NORAPS  for  short-range  forecasting  of  elevated  trapping  layers;  however,  NABL’s 
ability  in  surface  ducting  assessment  was  found  to  be  limited.  It  is  recommended  that  the  NABL 
luodel  be  upgraded  to  operational  status  so  that  its  use  in  refractivity  forecasting  and  in 
applications  programs  which  require  high  resolution  boundary  layer  information  can  be  exploited. 
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AN  EVALUATION  OF  THE  NAVY  ATMOSPHERIC  BOUNDARY  LAYER  MODEL 


1.  INTRODUCTION 

In  response  to  the  Navy’s  critical  needs  for  short-range  forecasts  in  the  marine  boundary 
layer,  the  Navy  Atmospheric  Boundary  Layer  (NABL)  model  has  been  developed.  This  model 
has  successfully  transitioned  from  exploratory  to  advanced  development  and  undergone  extensive 
testing  in  both  research  and  operational  environments.  In  order  to  transition  the  NABL  model 
to  operational  implementation  at  FNOC  (Fleet  Numerical  Oceanography  Center),  a  validation  test 
report  is  required.  This  NOARL  Technical  Note  presents  the  results  from  a  test  performed  to 
validate  the  NABL  model. 

The  verification  of  the  NABL  model  is  by  direct  comparison  with  radiosondes.  Statistical 
differences  between  NABL  model  forecast  ambient  and  dewpoint  temperatures  and  radiosondes 
are  calculated.  An  assessment  of  the  NABL  model’s  ability  to  forecast  refractive  structure  is 
established,  based  on  comparisons  of  model  and  radiosonde  refractivity  profiles.  In  particular, 
a  determination  is  made  as  to  how  well  the  NABL  model  is  able  to  assess  and  forecast  the  most 
important  anomalous  propagation  phenomenon  to  impact  naval  operations  -  ducting.  In  order  to 
assess  the  degree  of  skill  and  usefulness  of  the  NABL  model,  direct  comparisons  are  made 
among  NABL  model  forecasts  and  analogous  forecasts  from  the  Navy  Operational  Global 
Atmospheric  Prediction  System  (NOGAPS)  and  a  specially  implemented  version  of  the  Navy 
Operational  Regional  Atmospheric  Prediction  System  (NORAPS). 

2.  NABL  MODEL  DESCRIPTION 

The  NABL  model  is  a  one-dimensional  numerical  model  designed  to  make  high 
resolution,  short-range  (up  to  36  hr)  predictions  in  the  atmospheric  boundary  layer.  Its  main 


1 


attributes  are  sophisticated  boundary  layer  physics,  moist  thermodynamics  and  a  very  higlr 
vertical  resolution.  The  model  contains  prognostic  equations  for  the  mean  variables  -  vhe  wind 
components,  liquid  water  potential  temperature  and  total  moisture.  Prognostic  equations  for 
turbulent  kinetic  energy,  temperature  and  moisture  variances,  and  their  covariances,  are  given  by 
a  second  order  closure  turbulence  parameterization.  The  model’s  moist  thermodynamics 
calculations  include  liquid  water  content  and  fractional  cloudiness;  precipitation  is  calculated 
based  upon  an  assumed  cloud  droplet  distribution  function.  A  detailed  radiation  scheme 
computes  fluxes  of  long  and  shortwave  radiation  and  heating/cooling  rates.  The  NABL  model 
grid  contains  40  points  within  4.5  km  of  the  surface  (i.e.,  below  ~580  mb),  with  spacing 
compacted  toward  the  surface  such  that  6  points  lie  within  the  first  100  m.  Vertical  resolution 
ranges  from  5  m  at  the  surface  to  150  m  above  1.5  km.  Further  details  of  NABL  model  physics 
are  given  by  Burk  (1989)  and  Burk  and  Thompson  (1989). 

A  unique  feature  of  the  NABL  model  is  its  coupling  to  the  Navy’s  operational  large-scale 
models  -  the  NOGAPS  and  the  NORAPS  (See  Appendix  A  for  descriptions  of  these  models). 
Operationally,  the  NABL  model  is  initialized  witli  a  vertical  profile  and  surface  information  from 
a  "host"  large-scale  model  (either  NOGAPS  or  NORAPS).  In  general,  NABL  initial  gridpoint 
data  are  obtained  by  interpolation  from  the  large-scale  model  sigma  surfaces;  however,  for  those 
NABL  gridpoints  below  the  large-scale  model’s  lowest  computational  level,  initial  NABL 
temperature  and  humidity  values  are  set  equal  to  those  at  the  lowest  surface.  If  necessary,  NABL 
initial  profiles  are  modified  to  eliminate  superadiabatic  or  supersaturated  layers.  During 
execution,  previously  saved  adiabatic  tendencies,  at  hourly  forecast  intervals  from  the  host  model, 
are  used  to  represent  horizontal  and  vertical  processes  not  modeled  by  NABL.  For  this  study, 
the  initial  profiles  and  surface  information,  as  well  as  the  tendencies,  from  the  host  NOGAPS  (or 
NORAPS)  model  are  those  at  the  nearest  NOGAPS  (or  NORAPS)  gridpoint  to  the  specified 
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NABL  forecast  location.  In  general,  the  NABL  model  is  quite  sensitive  to  the  forecast  location 
type  (land  or  water),  which  is  uniquely  specified  by  the  model’s  ground  w'etness  parameter  for 
that  location.  For  land  points,  the  degree  of  diurnal  variation  within  the  boundary  layer  is  very 
pronounced,  whereas,  for  water  points,  it  is  much  less  so. 

3.  VERinCATlON 

3.1  Data 

As  verification  for  NABL  model  forecasts,  routine  radiosondes  from  five  island  sites  - 
Sable  Is.,  CAN  (YSA);  Cape  Hatteras,  NC  (HAT);  Bermuda,  U.K.  (XKF);  Nassau,  Bahamas, 
U.K.  (YNN);  and  Key  West,FL  (EYW)  -  and  one  land  station  (Greensboro,  NC  (GSO))  were 
utilized  (Figure  1).  With  the  exception  of  XKF,  all  island  station  elevations  (i.e.,  radiosonde 
launch  heights)  were  within  a  few  meters  of  sea  level  (see  Table  1).  All  sites  (except  YNN) 
launch  sondes  twice  daily  (OOZ  and  12Z);  YNN  has  only  a  12Z  launch  schedule.  Over  the  three 
week  period  of  study  (11  April  -  01  May  1991),  radiosonde  availability  (from  FNOC  data 
records)  was  near  90%  at  all  sites  except  YNN,  where  only  75%  of  scheduled  sondes  were 
available.  For  comparison  with  NABL  forecast  profiles,  only  lower  tropospheric  sounding  data 
(surface  to  ~600  mb)  were  required.  Radiosonde  data  included  both  standard  and  significant 
levels;  each  radiosonde  was  gross  error  checked  to  ensure  some  degree  of  accuracy  and 
consistency. 

Two  groups  of  NABL  model  forecasts  were  compared  to  soundings;  one,  NABL  forecasts 
coupled  to  NOGAPS  (hereafter  designated  NABL(NG))  and  two,  those  coupled  to  NORAPS 
(NABL(NR)).  In  both  cases,  NABL  model  profiles  were  loosely  co-located  with  the  selected 
radiosonde  sites;  that  is,  the  model-derived  profile  for  a  location  is  given  by  the  model  profile 
at  the  nearest  NOGAPS  (or  NORAPS)  gridpoint.  In  this  study,  for  any  particular  model  run,  two 
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Figure  1.  NORAPS  forecast  domain  and  radiosonde  locations  used  in  study. 
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Table  1.  Model  terrain  and  radiosonde  station  heights  (in  meters)  at  selected  stations:  YSA 
(Sable  Is.,  CAN);  HAT,  HTL  (Cape  Hatteras,  NC);  GSO  (Greensboro,  NC);  XKF  (Bermuda, 
U.K.);  YNN  (Nassau,  Bahamas,  U.K.);  and  EYW  (Key  West,  FL).  HAT  (HTL)  is  considered 
a  NABL  model  water  (land)  location.  No  NABL(NG)  height  is  given  for  GSO;  NOGAPS  data 
were  not  used  at  this  location. 


STATION 

RADIOSONDE 

NABL(NR) 

NABL(NG) 

YSA 

4 

0 

4 

HAT 

2 

0 

1 

HTL 

2 

0 

97 

GSO 

277 

233 

- 

XKF 

27 

0 

-1 

YNN 

2 

0 

4 

EYW 

2 

0 

4 

distinct  NABL  profiles  were  available  at  Cape  Hatteras,  NC  -  one  considered  to  be  at  a  water 
point  (HAT),  the  other  over  land  (HTL).  All  other  NABL  forecasts,  with  the  exception  of  GSO 
(a  land  point),  were  considered  to  be  at  water  points.  Inspection  of  Table  1  indicates  that  surface 
terrain  heights  slightly  exceed  zero  at  several  NABL(NG)  ocean  locations.  In  general,  vertical 
offsets  between  radiosondes  and  corresponding  model  profiles  are  not  large;  offsets  >  20  m  only 
occur  at  XKF,  HTL  (for  NABL(NG))  and  GSO.  NABL  forecasts  were  made  at  both  OOZ  and 
12Z  start  times,  v/ith  forecasts  at  tau  =  0,  12  and  24.  (Note:  Tau  refers  to  the  model  forecast 
period  in  hours;  tau  =  0  is  an  initial  (analysis)  field.) 

For  statistical  comparisons  with  NABL  model  profiles,  NORAPS  model-derived  profiles 
(at  tau  =  12  and  24)  were  extracted  for  all  study  locations.  Additionally,  these  NORAPS  profiles 
(and  corresponding  NOGAPS  profiles  at  tau  =  0  and  12)  were  used  for  comparisons  of  large- 
scale  and  NABL  model  capabilities  for  refractivity  assessment  and  forecasting.  The  NOGAPS 
model-derived  profiles  used  in  this  study  were  based  on  data  from  the  six  lowest  model 
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computational  levels  (i.e.,  below  ~2.3  km  when  the  surface  pressure  is  1  atm).  NOGAPS 
(NORAPS)  terrain  heights  correspond  to  NABL(NG)  (NABL(NR))  station  values  given  in 
Table  1. 

3.2  Techniques 

This  validation  of  the  NABL  model  is  based  on  two  main  criteria;  one,  model  error 
statistics  and  two,  capabilities  for  assessment/forecasting  of  refractivity  structure,  in  particular, 
the  occurrence  of  ducting.  Additionally,  a  visual,  qualitative  evaluation  is  made  of  NABL 
forecast  quality  in  regards  to  surface  layer  features.  The  two  statistics  utilized  in  this  validation 
are  the  bias  and  the  root-mean-square  (RMS)  difference,  and  are  computed  for  the  temperature 
and  dewpoint  temperature.  To  calculate  these  statistics,  model  temperature  and  dewpoint 
temperature  data  are  interpolated,  assuming  a  linear  variation  with  the  log  of  the  pressure,  to  all 
reporting  levels  (standard  and  significant)  of  the  corresponding  radiosonde  which  are  below  the 
NABL  model’s  highest  forecast  level  (4.5  km  or  -580  mb).  The  temperature  and  dewpoint 
temperature  differences  between  the  model  and  radiosonde  at  all  applicable  levels  are  used  collec¬ 
tively  to  compute  integral  approximations  of  the  bias  and  RMS  difference.  Unlike  pomt-by-pomt 
approximations  (which  equally  weigh  all  data  in  averaging),  integral  approximations  of  error 
statistics  assign  proportionally  more  (less)  weight  in  averaging  to  widely  (closely)  spaced  data 
within  the  sounding.  Given  estimates  of  bias  and  RMS  difference  between  each  radiosonde  and 
the  corresponding  model  profile,  average  temperature  and  dewpoint  temperature  error  statistics 
can  be  readily  computed  for  any  one  station  or  group  of  stations  (e.g.,  those  at  water  points),  or 
for  a  particular  forecast  tau  or  verification  time.  In  this  study,  average  error  statistics  are 
compared  among  NABL(NG),  NABL(NR)  and  NORAPS  forecasts. 
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Given  conventional  radiosonde  or  model  profile  data,  an  assessment  of  ducting  can 
conveniently  be  made  in  terms  of  the  modified  refractivity  M,  which  is  expressed  as, 

M  =  77.6p/T  +  3.73x10^  e/T^  +  0.157z  , 
where  p  is  the  atmospheric  pressure  in  millibars, 

T  is  the  temperature  in  Kelvin, 

e  is  the  vapor  pressure  in  millibars,  and 

z  is  the  height  above  the  sea  surface  in  meters. 

Whenever  M  decreases  with  altitude  within  a  layer  (dM/dz  <  0),  a  trapping  layer  is  present,  and 
enhanced  microwave  propagation  occurs  within  the  duct  below  the  top  of  the  trapping  layer  for 
small  incidence  angles.  A  duct  can  be  either  surface  or  elevated;  a  surface-based  duct  can  result 
from  an  elevated  trapping  layer  (ETL)  provided  that  the  value  of  M  at  the  top  of  the  ETL  is  less 
than  the  value  of  M  at  the  surface. 

Radiosonde  M-profiles  were  computed  from  mandatory  and  significant  sonde  levels  within 
the  lower  troposphere.  This  required  the  conversion  of  dewpoint  temperatures  at  reporting  sonde 
levels  to  vapor  pressures,  and  the  determination  of  sounding  heights  at  significant  levels  through 
application  of  the  hypsometric  formula.  In  general,  the  vertical  resolution  of  any  particular 
radiosonde-derived  M-profile,  which  is  a  function  of  the  number  of  significant  levels,  is  in 
proportion  to  the  complexity  of  lower  tropospheric  meteorological  conditions.  On  the  other  hand, 
the  vertical  resolution  of  a  NABL  model  M-profile  is  invariant  and  that  of  a  NOGAPS  or 
NORAPS  profile  is,  for  all  practical  purposes,  also  fixed.  In  this  study,  differences  in  vertical 
resolution  between  radiosonde  and  model  M-profiles  are  not  resolved;  comparisons  of  ducting 
occurrence  among  model  forecasts  and  soundings  will  be  based  on  actual  model  and  radiosonde 
resolution. 
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A  cursory  examination  of  all  NABL  forecasts  available  for  this  study  indicates  a  large 
number  with  veiy  shallow  surface  ducts.  In  general,  very  shallow  surface  ducts  cannot  be 
resolved  by  radiosondes  and  thus,  should  be  eliminated  from  comparisons.  To  accomplish  this, 
an  analysis  criterion  was  set  to  eliminate  all  very  shallow  NABL  surface  ducts.  This  criterion 
states  that  a  NABL  surface  duct  will  be  considered  to  have  occurred  if  a  model  profile  surface- 
based  duct  height  >  40  m  occurs. 

In  regards  to  elevated  ductin*^,  an  elevated  trapping  layer  will  be  considered  to  occur  for 
any  profile  (radiosonde  or  model-derived)  provided  that  the  top  of  any  one  ETL  is  at  least  40  m 
above  the  surface  and  no  more  than  2.3  km  above  the  surface.  The  upper  limit  for  ETL 
occurrences  essentially  applies  only  tu  radiosondes,  since  model  (NOGAPS,  NORAPS  or  NABL) 
ETL  occurrences  do  not  generally  occur  above  1.5  km.  The  occurrence  of  multiple  ETLs  in  a 
profile  will  be  counted  as  one  occurrence.  A  "correct"  model  forecast  of  an  ETL  is  not  restricted 
by  any  condition  which  would  require  the  forecast  ETL  to  be  located  within  a  certain  specified 
range  of  the  observed  ETL. 

The  verification  of  any  particular  model’s  ability  to  assess  or  forecast  ducting  will  be 
based  primarily  on  two  simple  contingency  table  statistics,  the  prefigurance  (P^)  and  the 
postagreement  (P^).  Given  any  particular  event  (viz.,  ducting),  the  prefigurance  is  the  capability 
of  correctly  forecasting  that  event,  and  is  defined  as  the  number  of  correct  (model)  forecasts 
divided  by  the  number  of  observed  (radiosonde)  occurrences.  Postagreement  is  the  reliability  of 
the  forecasts  that  were  issued,  and  is  defined  as  the  number  of  correct  (model)  forecasts  divided 
by  the  number  of  forecasts  issued. 
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4. 


COMPARISON  RESULTS 


The  evaluation  of  a  model’s  meteorological  and  refractive  stru  ctures  by  direct  comparison 
with  soundings  (as  per  this  study)  is  certainly  not  a  fully  adequ^fc  method.  A  radiosonde 
represents  only  one  "snapshot,"  or  realization,  from  a  turbulent  medium,  providing  point 
measurements  which  define  significant  temperature  and/or  moisture  locations  in  the  vertical.  On 
the  other  hand,  a  model  nrcille  reflects  a  volume-averaged  atmospheric  structure,  one  where 
sharp  gradients  are  often  not  ''-eh  analyzed.  Particularly  in  coastal  regions,  this  fundamental 
difference  can  lead  to  a  direr  comparison  o,  maritime  radiosonde  with  a  continental  co-located 
model  profile.  Differences  in  vei^LoI  resolution,  and  vertical  offsets  between  radiosondes  and 
correspond.'ng  model  profil  •  adversely  affect  direct  comparisons. 

4.1  Stall 

L'  '..his  stu^'.y,  the  :'^mp?’'ature  and  dewpoint  temperature  serve  as  the  basic  atmospheric 
paramet  ;...  fo:  modei  val.-dadu.'  '.-•.like  the  case  for  temperature,  statistical  results  based  on 
dewpoint  temperature  are  difficult  to  interpret.  This  moisture  parameter  tends  to  be  overly 
sensitive,  especially  away  from  the  surface,  where  large  differences  in  dewpoint  temperature  only 
equate  to  small  differences  in  actual  water  vapor  content. 

A  comparison  ’^^ORAPS,  NABL(NR)  and  NABL(NG)  error  statistics  for  temperature, 
ba.Sf  d  on  all  available  water  point :  ;  ,asts  {i.t.,  YSA,  HAT,  XKF,  YNN  and  EYW),  is  presented 

in  Figure  2.  These  model  statistics  are  based  on  a  total  of  130,  135  and  136  coincidental 
forecasts,  at  tau  =  0,  12  and  24,  respectively.  NABL(NR)  and  NABL(NG)  mean  RMS 
differences  are  seen  to  increase  roughly  a  degree  from  the  initial  (analysis)  time  to  tau  =  24. 
NABL  values  at  tau  =  12  and  24  are  only  a  few  tenths  of  a  degree  greater  than  values  for 
NORAPS.  These  slight  differences  practically  disappear  when  one  considers  median  RMS 
differences;  in  particular,  median  RMS  temperature  differences  among  models  differ  by  no  more 
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Figure  2.  Comparison  of  NORAPS,  NABL(NR)  and  NABL(NG)  mean  RMS  difference  and  bias 
statistics  for  temperature,  derived  from  all  available  water  point  forecasts,  at  tau  =  0,  12  and  24. 
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than  a  tenth  of  a  degree  at  tau  =  12  and  24.  Mean  NABL  temperature  biases  are  not  significant, 
and  are  within  a  few  tenths  of  a  degree  of  zero  over  the  entire  forecast  period.  On  the  other 
hand,  the  NORAPS  model  ejthibits  a  markedly  cold  bias  (forecast  lower  than  observed)  which 
increases  about  a  half  degree  from  tau  =  12  to  tau  =  24.  A  total  of  almost  90%  of  NORAPS 
forecasts  at  tau  =  24  had  cold  biases.  NABL(NR)  forecasts  did  not  indicate  any  decided  bias 
(warm  or  cold)  in  terms  of  frequency;  the  percentage  of  NABL(NG)  forecasts  with  a  cold  bias 
increased  only  slightly  over  the  forecast  period,  from  55%  at  tau  =  0  to  63%  at  tau  =  24. 

Figure  3  depicts  the  NABL  and  NORAPS  mean  RMS  difference  and  bias  statistics  for 
dewpoint  temperature,  derived  from  all  available  water  point  forecasts.  In  general,  the  RMS 
differences  and  biases  are  quite  large,  reflecting  both  the  sensitivity  of  dewpoint  temperature  as 
a  moisture  parameter  and  the  lack  of  a  moisture  analysis  with  either  NOGAPS  or  NORAPS. 
Mean  NABL  RMS  differences  increase  roughly  2°C  over  the  forecast  period.  Differences 
between  NORAPS  and  the  NABL  model  are  not  great,  and  increase  from  about  a  half  degree  at 
tau  =  12  to  about  one  degree  at  24  hours.  Mean  dewpoint  temperature  biases  for  the  NABL 
model  are  slightly  negative  initially,  but  increase  rapidly  over  the  forecast  period.  Although 
difficult  to  interpret,  this  marked  increase  in  negative  bias  for  dewpoint  temperature  suggests  that 
the  NABL  model’s  initial  moisture  distribution  tends  to  dry  out  over  time.  The  NORAPS 
negative  dewpoint  temperature  biases,  which  are  less  severe  than  those  for  NABL,  are  seen  to 
increase  only  slightly  from  tau  =  12  to  tau  =  24.  The  difference  between  these  NORAPS  biases 
and  corresponding  NABL  biases  increases  considerably  over  this  forecast  time  interval.  Finally, 
when  median  values  are  considered,  both  the  devv^joint  temperature  RMS  difference  and  negative 
bias  for  NABL(NG),  at  tau  =  12,  are  actudly  slightly  less  than  the  corresponding  values  for 
NORAPS. 
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Figure  3.  Comparison  of  NORAPS,  NABL(NR)  and  NABL(NG)  mean  RMS  difference  and  bias 
statistics  for  dewpoint  temperature,  derived  from  all  available  water  point  forecasts,  at  tau  =  0, 
12  and  24. 
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A  direct  comparison  of  model  land  and  water  point  forecasts  is  available  only  at  one  site  - 
Cape  Hatteras,  NC.  Table  2  presents  NORAPS,  NABL(NR)  and  NABL(NG)  mean  RMS  differ¬ 
ence  and  bias  statistics  at  HAT  (water  point)  and  HTL  (land  point).  Statistics  given  in  this  table 
are  based  on  only  30  forecasts  for  each  tau,  and  thus  lack  a  strong  statistical  base.  At  each  fore¬ 
cast  period,  temperature  and  dewpoint  temperature  mean  RMS  differences  for  all  models  are 
larger  for  land  forecasts.  The  magnitude  of  these  differences  between  land  and  water  forecasts 
is  most  noticeable  for  NABL(NG),  and  exceed  1°C  at  tau  =  24.  Mean  temperature  biases  for 
NORAPS  and  NABL  also  show  a  common  trend  throughout  the  forecast  period,  that  is,  more 
negative  for  land  po.in^  fore'^asts  NABL(NG)  again  presents  the  greatest  difference  between  land 
and  water  forecrs'^  «h(--av  1.4°C  at  all  forecast  intervals.  With  the  exception  of  NORAPS  at 
tau  =  12  and  NABL(i4R)  at  tau  =  0,  dewpoint  temperature  biases  are  also  larger  (i.e.,  more 
negative)  for  land  forecasts.  Differences  between  NABL(NG)  land  and  water  point  model 
statistics  could  possibly  be  influenced  by  differences  in  model  terrain  height  in  addition  to 
different  forecast  surfaces  (land  or  water).  Unlike  NABL(NG)  water  point  forecasts,  which  begin 
near  sea  level,  NABL(NG)  land  point  forecasts  start  at  almost  100  m  above  the  surface;  radio¬ 
sonde  data  below  this  height  do  not  enter  into  computations  of  model  statistics. 

In  order  to  examine  temporal  differences  in  model  statistics,  water  point  forecasts  were 
divided  into  two  groups  at  each  forecast  period  -  those  which  verify  at  OOZ,  and  those  at  12Z. 
Results  from  these  statistical  comparisons  revealed  no  noticeable  differences.  Excepting  the 
previous  comparison  of  HAT  and  HTL  model  error  statistics,  comparisons  between  individual 
stations  are  not  pursued  in  this  study  due  to  the  relatively  low  sample  size,  which  adversely 
affects  the  reliability  of  statistical  comparisons. 
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Table  2.  (a)  Mean  RMS  difference  and  b'os  statistics  for  temperature  (in  ®C),  derived  from 
NORAPS,  NABL(NR)  and  NABL(NG)  water  (HAT)  and  land  (HTL)  point  forecasts  at  Cape 
Haneras,  NC,  at  tau  =  0,  12  and  24.  (b)  Same  as  (a),  except  for  dewpoint  temperamre. 


(a) 

TEMPERATURE 

RMS 

HjB 

TAU 

NR 

NABL(NR) 

NABL(NG) 

00 

1.34 

MSM 

■ai 

-  .02 

HAT 

00 

1.49 

HbI 

-1.33 

HTL 

■9 

1.44 

■H 

1.58 

n^n 

-.29 

-  .07 

HAT 

■a 

1.64 

2.51 

-.53 

-1.47 

HTL 

Bl 

n 

WBM 

2.03 

1 

-  .99 

-.18 

.26 

HAT 

El 

miM 

3.04 

-1.30 

-.34 

-1.14 

HTL 

(b) 

DEWPOINT  TEMPERATURE 

RMS 

TAU 

NR 

NABL(NR) 

NABL(NG) 

IE 

IQE 

4.87 

WEM 

.23 

HAT 

■1 

5.73 

.11 

HTL 

n 

5.89 

6.21 

5.75 

IB 

-  .82 

HAT 

Is 

5.95 

6.45 

6.44 

wm 

-1.81 

HTL 

Bl 

n 

6.62 

f 

-  .92 

-1.92 

HAT 

Bl 

7.13 

-  .97 

-2.98 

iiE&Dm 

HTL 

4.2  Refractivity 

Due  to  model  limitations,  the  surface  ducting  assessment  capabilities  of  the  NOGAPS  and 
NORAPS  are,  for  all  practical  purposes,  nil.  Table  3  presents  the  prefigurance  and  postagreement 
statistics  for  the  NABL  model  for  surface  ducting  assessment.  A  noticeable  feature  of  these  data 
are  the  low  number  of  observed  as  well  as  forecast  surface  ducts,  most  especially  at  water  point 
sites.  Although  statistically  sound  conclusions  cannot  be  drawn  from  small  sample  sizes,  results 
in  Table  3  do  suggest  that  the  usefuhiess  of  NABL  model  forecast  information  in  defining  surface 
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Table  3.  (a)  Surface  ducting  prefigurance  and  postagreement  statistics  for  NABL(NG)  and 
NABL(NR),  derived  from  all  available  water  point  forecasts.  NABL  model  subscripts  indicate 
forecast  tau.  (b)  Same  as  (a),  except  for  all  available  land  point  forecasts.  NABL(NG)  forecasts 
are  only  available  at  HTL.  Note  that  postagreement  is  undefined  for  NABL(NG)q. 


(a)  MODEL 

NO.  DATA 

PREFIGURANCE 

POSTAGREEMENT 

NABL(NG)o 

128 

0/9  =  .000 

0/1  =  .000 

NABL(NR)o 

128 

1/9  =  .111 

1/1  =  1.000 

NABL(NG)i2 

133 

2/10  =  .200 

2/3=  .667 

NABL(NR)i2 

133 

0/10  =  .000 

0/4=  .000 

NABL(NG)24 

134 

2/11  =  .182 

2/7=  .286 

NABL(NR)24 

134 

0/11  =  .000 

0/10=  .000 

(b)  MODEL 

NO.  DATA 

PREFIGURANCE 

POSTAGREEMENT 

NABL(NG)o 

29 

0/7  =  .000 

0/0=  (UNDF) 

NABL(NR)o 

59 

0/14  =  .000 

0/1  =  .000 

NABL(NG)j2 

30 

2/7  =  .286 

2/9=  .222 

NABL(NR)i2 

61 

3/14  =  .214 

NABL(NG)24 

30 

2/7  =  .286 

2/11=  .182 

NABL(NR)24 

61 

3/14  =  .214 

3/16=  .188 

ducting  occurrences  is  quite  limited.  This  conclusion  is  the  same  as  that  drawn  in  a  previous, 
more  extensive  study  on  forecast  model  capabilities  in  ducting  assessment  (Vogel,  1991). 

In  that  earlier  study,  Vogel  found  evidence  that  NABL  12  and  24  hour  land  forecasts  had 
markedly  higher  capabilities  and  reliabilities  at  the  OOZ  verification  time  than  at  VT  12Z.  A 
careful  examination  of  available  NABL  (NG  and  NR)  forecasts  at  both  land  locations  (HTL  and 
GSO)  also  shows  a  similar  feature.  These  temporal  differences  in  NABL  12  and  24  hour  land 
forecasts  were  likely  caused  by  a  bias  in  the  model’s  surface-energy  budget  equation  for  ground 
temperature.  Finally,  although  data  are  limited,  a  direct  comparison  of  land  and  water  point 
forecasts  -  at  Cape  Hatteras  -  reveals  that  NABL  (NG  and  NR)  12  and  24  hour  surface  duct 
forecasts  are  more  likely  over  land  than  water;  this  finding  is  in  agreement  with  the  previous 
study  by  Vogel. 
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Performance  statistics  of  the  NOGAPS,  NORAPS  and  the  NABL  model  for  the 
assessment  and/or  forecasting  of  elevated  trapping  layers,  derived  from  all  available  water  point 
data,  are  presented  in  Figure  4.  NOGAPS,  based  on  a  limited  subset  of  data  at  tau  =  0  and  12, 
is  found  to  be  completely  useless  for  ETL  assessment.  On  the  contrary,  NORAPS  12  and  24 
hour  forecasts  show  some  capability,  and  quite  good  forecast  reliability.  In  the  previous  study 
by  Vogel,  no  ETL  assessment  utility  was  found  for  NORAPS.  Factors  which  likely  contributed 
to  the  significant  improvement  in  NORAPS ’s  performance  include  improved  model  physics  (and 
computer  code),  a  higher  model  horizontal  resolution  (40  km  versus  100  km)  and  the  inclusion 
of  a  12  hr  incremental  update  cycle.  At  tau  =  0,  NABL(NR)  shows  considerably  better  forecast 
capability  than  NABL(NG);  in  addition,  NABL(NR)q  had  a  considerably  greater  number  of 
reliable  forecasts.  NABL(NR)  12  and  24  hour  forecast  capabilities  (both  near  0.4)  are  noticeably 
better,  especially  at  tau  =  24,  than  those  for  NORAPS.  Moreover,  although  NABL  12  and  24 
hour  forecast  reliabilities  are  similar  to  those  for  NORAPS,  the  actual  number  of  correct  forecasts 
(and  forecasts  issued)  is  somewhat  greater  for  NABL(NR).  In  general,  NABL(NG)  12  and  24 
hour  forecasts  are  not  good  as  those  for  NABL(NR);  moreover,  while  NABL(NG)  12  and  24  hour 
ETL  forecast  capabilities  are  comparable  to  those  for  NORAPS,  forecast  reliabilities  appear 
somewhat  less.  The  better  vertical  and  horizontal  resolution  of  NORAPS  compared  to  NOGAPS 
is  likely  the  key  contributing  factor  as  to  why  NABL(NR)  ETL  forecasts  are  better  than  those 
based  on  NOGAPS  information.  Finally,  although  NORAPS  and  NABL(NG  and  NR)  perform¬ 
ance  statistics  at  tau  =  12  appear  slightly  better  than  those  for  24  hour  forecasts,  these  differences 
are  not  statistically  significant  based  on  the  sample  size.  Interestingly,  in  the  previous  study  by 
Vogel,  NABL(NR)  24  hour  performance  statistics  were  slightly  better  than  those  at  12  hr; 
however,  in  that  case,  NABL  initial  profiles  did  not  benefit  from  a  12  hr  NORAPS  update  cycle. 
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PREFIGURANCE  (Pf) 


NOGAPS  NGq  68  0/35  =  .000  0/0  =  (UNDP) 

NABL(NG)o  130  2/64  =  .031  2/3  =.667 

NABL(NR)q  130  13/64  =  .203  13/20  =  .650 

NOGAPS  NGj2  52  0/30  =  .000  0/0  =  (UNDP) 

NORAPS  NR|2  135  22/68  =  .324  22/29  =  .759 

NABL(NG)j2  135  19/68  =  .279  19/30  =  .633 

NABL(NR)j2  135  27/68  =  .397  27/40  =  .675 

NORAPS  NR24  136  17/69  =  .246  17/26  =  .654 

NABL(NG)24  136  16/69  =  .232  16/29  =  .552 

NABL(NR)24  136  27/69  =  .391  27/42  =  .643 


Pigure  4.  Elevated  trapping  layer  prefigurance  and  postagreement  statistics  for  the  NOGAPS, 
NORAPS  and  the  NABL  model,  derived  from  ail  available  water  point  forecasts.  Subscripts 
indicate  forecast  tau.  Note  that  postagreement  in  undefined  for  NOGAPS  at  tau  =  0  and  12. 


For  this  validation,  the  capabilities  of  the  NORAPS  and  the  NABL(NG  and  NR)  models  to 
assess  relatively  deep  (>  100  m  thick)  elevated  trapping  layers  were  also  determined.  In  general, 
these  results  are  little  changed  from  those  based  on  all  ETL  occurrences  (Fig.  3).  Finally,  given 
the  low  sample  size,  a  direct  comparison  of  NABL(NG  and  NR),  and  NORAPS,  land  and  water 
point  forecasts  at  Cape  Hatteras  showed  no  significant  differences  in  ETL  assessment 
performance. 

4.3  Qualitative  Evaluation  of  Forecast  ABL 

A  purely  statistical  approach  to  model  evaluation  can  produce  misleading  conclusions, 
even  when  both  RMS  errors  and  forecast  bias  are  taken  into  consideration.  For  example,  a  low 
RMS  error  or  bias  statistic  does  not  provide  any  information  concerning  features  such  as  mixed 
layer  depth  or  lapse  rate  that  may  be  of  tactical  interest. 

t  visual,  qualitative  evaluation  was  made  of  the  forecasts  generated  by  the  NABL  model 
and  NORAPS  against  the  verifying  raobs.  Particular  attention  was  given  to  forecast  quality  in 
the  surface  layer.  "Good"  forecast  features  include  a  correct  depiction  of  the  lapse  rate  (stable 
or  unstable)  in  the  surface  layer.  If  there  is  a  surface-based  inversion,  does  the  model  accurately 
forecast  the  inversion  depth?  If  the  lapse  rate  in  the  surface  layer  is  unstable  with  a  mixed  layer, 
is  the  depth  of  the  well-mixed  layer  accurate?  Does  the  model  forecast  a  cloud  at  the  top  of  the 
mixed  layer? 

Inspection  of  selected  forecasts  from  the  April  1991  model  simulations  reveals  some  of 
the  strengths  and  weaknesses  of  the  models.  NABL  with  NOGAPS  tendencies  forecasts  the  most 
frequent  and  deepest  mixed  layers,  but  is  also  more  likely  to  over-predict  the  mixed  layer  depth 
and  to  predict  a  mixed  layer  when  the  observed  near-surface  lapse  rate  is  stable.  It  is  possible 
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that  the  relatively  coarse  vertical  resolution  of  the  NOGAPS  temperature  tei'<’enci  vhen 
interpolated  to  the  finer  NABL  model  grid  produces  superadiabatic  lapse  rate-J  that  are  removed 
in  the  NABL  model  by  vertical  mixing  in  unrealistically  deep  layers. 

The  NABL  model  does  somewhat  better  at  forecasting  mixed  layer  structure  when  the 
large-scale  tendencies  are  obtained  from  NORAPS  rather  than  NOGAPS.  However,  as  mentioned 
in  Section  4.1,  in  both  configurations  the  NABL  model  does  have  a  dry  bias,  which  can  have 
adverse  effects  on  fog  or  cloud  development.  This  can  be  significant,  because  cloud-top 
radiational  cooling  is  a  major  forcing  mechanism  in  the  deepening  of  well-mixed  layers.  The 
cause  of  the  dry  bias,  which  may  be  due  to  effects  of  cumulus  convection  in  the  large-scale 
tendency  terms,  is  under  investigation. 

Excessive  moisture  in  mixed  layers  is  seen  in  the  NORAPS  forecasts  at  tlie  sites  selected 
for  the  April  1991  evaluation  period.  Clouds  and  saturated  layers  are  often  too  deep  in  the 
NORAPS  forecasts.  In  terms  of  predicting  mixed  layer  structure,  the  NABL(NR)  and  NORAPS 
forecasts  are  of  roughly  equal  quality.  Again,  it  is  the  NABL  model  ability  to  forecast  elevated 
trapping  layers  that  makes  its  forecast  of  value  beyond  the  NORAPS  forecast. 

4.4  A  Few  Words  About  Climatology 

Given  an  assessment  of  model  forecast  performance,  it  is  often  appropriate  to  determine 
if  climatology  might  also  provide  useful  guidance  to  the  forecaster.  For  example,  the  NABL 
model  skill  at  predicting  elevated  trapping  layers,  w'hich  was  discussed  in  Section  4.2,  can  be 
compared  to  a  simple  yes/no  approach  using  climatology. 

Ehiring  the  verification  period  of  April  1991,  the  number  of  elevated  trapping  layers 
observed  at  Key  West  had  an  overall  occurrence  rate  of  about  0.65  (19  of  30  raobs),  which  is 
slightly  above  the  expected  climatological  rate  of  0.55  (Patterson,  1982).  If  only  this  one  statistic 
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was  involved  in  a  decision,  an  elevated  trapping  layer  could  be  forecast  every  time  with  ~65% 
reliability.  The  climatology  postagreement  score  would  be  ~0.65  (19/30)  and  the  climatology 
prefigurance  (skill)  score  would  be  1.0  (19/19).  There  would  be  11  "false  alarm"  climatology 
ETL  forecasts. 

The  NABL(NR)  model  postagreement  score  for  Key  West  during  April  1991  was  near 
0.7,  so  it  can  be  stated  that  the  model  is  at  least  as  reliable  in  this  sense  as  climatology. 
However,  NABL  failed  to  predict  some  trapping  layers,  so  the  model  prefigurance  score  was  less 
than  climatology  at  this  location. 

Of  course  climatology  has  no  utility  for  forecasting  ETL  occurrence  if  values  are  less  than 
50%,  which  is  common  for  many  regions  and  times  of  year.  In  addition,  the  model  provides 
guidance  for  height  of  the  elevated  trapping  layer,  which  is  only  available  at  limited  locations 
or  on  highly  smoothed  charts  from  climatology. 

5.  CONCLUSIONS 

This  Technical  Note  presents  the  results  from  a  test  performed  to  validate  the  NABL 
model,  a  required  step  toward  operational  implementation  at  FNOC.  This  validation  of  the 
NABL  model  is  by  direct  comparison  with  radiosondes  and  is  based  on  two  main  criteria;  one, 
error  statistics  and  two,  capabilities  for  the  assessment  and  short-range  forecasting  of  lower 
tropospheric  ducting.  In  order  to  assess  the  degree  of  skill  of  the  NABL  model  in  predicting 
meteorological  and  refractive  structure,  NABL  model  forecasts  were  directly  compared  with  those 
from  a  specially  implemented,  high  horizontal  resolution  (40  km)  version  of  the  NORAPS. 

Model  error  statistics  are  calculated  as  integral  approximations  of  the  bias  and  RMS 
difference  between  model  profiles  and  radiosondes.  A  comparison  of  NABL  and  NORAPS 
statistics,  based  on  some  135  coincidental  model  forecasts  at  5  island  (water  point)  locations. 
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indicates  that  median  RMS  temperature  differences  for  these  models  differ  only  slightly  at  either 
the  12  or  24  hour  forecast  interval.  Mean  NABL  temperature  biases  are  found  to  be  close  to  zero 
at  all  forecast  intervals;  on  the  other  hand,  the  NORAPS  model  exhibits  a  pronounced  cold  bias 
which  increases  considerably  from  tau  =  12  to  tau  =  24.  In  general,  dewpoint  temperature  RMS 
difference  and  bias  statistics  for  the  NORAPS  and  NABL  model  are  quite  large;  this  reflects  both 
the  sensitivity  of  the  dewpoint  temperature  as  a  moisture  indicator  and  the  lack  of  a  moisture 
analysis  with  either  NOGAPS  or  NORAPS.  Mean  dewpoint  temperature  RMS  differences  for 
the  NORAPS  model  are  somewhat  lower  (by  ~0.5®C  at  tau  =  12,  ~1.0°C  at  tau  =  24)  than  corre¬ 
sponding  values  for  the  NABL  model.  Both  NORAPS  and  the  NABL  model  indicate  negative 
(i.e.,  dry)  dewpoint  temperature  biases;  not  only  are  the  NABL  model’s  12  and  24  hour  negative 
biases  more  pronounced,  they  are  also  found  to  increase  quite  rapidly  over  the  forecast  period. 
In  general,  a  direct  comparison  of  a  limited  number  (30)  of  land  and  water  point  forecasts 
indicates  somewhat  larger  (more  negative)  temperature  and  dewpoint  temperature  RMS 
differences  (biases)  over  land  than  water  for  both  the  NORAPS  and  the  NABL  model. 

With  regards  *0  refractivity,  evaluation  of  the  NABL  model  indicates  only  a  limited 
capability  and  reliability  in  the  assessment  and  short-range  forecasting  of  surface  ducts. 
Nonetlieless,  this  minor  utility  for  surface  ducting  assessment  is  a  modest  improvement  over  tlie 
virtually  useless  capabilities  of  the  NOGAPS  and  NORAPS.  Although  data  were  very  limited, 
an  examination  of  available  NABL  12  and  24  hour  surface  duct  forecasts  at  land  points  revealed 
temporal  differences  in  assessment  performance,  a  feature  previously  found  in  a  similar,  more 
extensive  study  (Vogel,  1991).  Model  performance  statistics  for  the  forecasting  of  elevated 
trapping  layers  show  a  decided  advantage  of  NABL(NG)  over  NOGAPS  at  tau  =  12.  While  both 
NORAPS  and  NABL(NR)  12  and  24  hour  ETL  forecast  reliabilities  are  very  good  (  near  0.7  ), 
NABL(NR)  12  and  24  hour  forecast  capabilities  (both  near  0.4)  are  considerably  better  than  those 
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NABL(NR)  ETL  performance  statistics  were  notably  higher  than  those  for  NABL(NG);  here,  the 
better  vertical  and  horizontal  resolution  of  NORAPS  compared  to  NOGAPS  is  likely  the  key 
contributing  factor.  Finally,  the  limited  data  available  in  this  study  did  not  reveal  any  significant 
enhancement  of  ETL  forecasting  performance  when  only  relatively  deep  (>  100  m  thick) 
elevated  trapping  layers  were  considered,  nor  show  any  significant  differences  in  NORAPS  or 
NABL  ETL  performance  statistics  at  land  and  water  locations. 

Statistical  comparisons  of  NABL  model  forecasts  of  temperature  and  moisture  with  those 
from  a  high  horizontal  resolution  version  of  NORAPS  do  not  in  themselves  indicate  a  superiority 
of  the  boundary  layer  model.  Athough  temperature  profiles  compare  favorably  with  those  for 
NORAPS,  negative  (dry)  moisture  biases  appear  more  severe  in  the  NABL  model.  When  model 
forecasts  of  temperature  and  moisture  are  combined  to  produce  refractivity  structure,  the  NABL 
model  shows  a  noticeable  advantage  in  forecasting  performance  over  NORAPS.  The  key 
ingredient  for  NABL’s  superiority  in  refractivity  forecasting  is  likely  the  model’s  very  high 
vertical  resolution.  Athough  not  evaluated  here,  the  NABL  model  high  resolution  profiles  of 
temperature  and  moisture  are  likely  to  be  of  substantial  forecasting  utility  for  boundary  layer 
quantities  of  direct  interest  besides  refractivity.  Given  the  Navy’s  present  critical  needs  for 
detailed  boundary  layer  information,  it  is  strongly  recommended  that  the  NABL  model  be  put 
into  immediate  operational  use. 
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APPENDIX  A  -  LARGE-SCALE  MODELS 


NOGAPS 

The  Navy  Operational  Global  Atmospheric  Prediction  System  (NOGAPS),  which  is  run 
twice  daily  (OOZ  and  12Z)  at  the  FNOC,  consists  of  a  multivariate  optimum  interpolation  (MVOI) 
analysis,  a  non-linear  normal  mode  initialization  scheme,  and  a  triangular  79  wave,  18  level 
spectral  forecast  model  (Barker  et  al.,  1988;  Hogan  et  al.,  1991).  The  MVOI  analysis,  which 
includes  a  6  hr  update  cycle  executed  4  times  daily,  produces  a  global  analysis  for  the  15 
standard  pressure  levels  (along  with  the  925  mb  level)  on  the  1.5  degree  Gaussian  grid  of  the 
spectral  forecast  model.  The  meteorological  variables  analyzed  are  the  geopotential  height  and 
the  wind;  the  virtual  temperature  is  derived  by  maintaining  an  approximate  hydrostatic  balance 
between  the  height  and  wind  fields.  Since  the  NOGAPS  currently  has  no  moisture  analysis,  the 
moisture  field  from  the  previous  model  run  becomes  the  initial  moisture  field  for  the  next 
forecast.  During  the  forecast  model  initialization  process,  meteorological  fields  are  interpolated, 
using  cubic  splines,  from  the  analysis  levels  to  the  model  grid.  The  spectral  forecast  model  uses 
a  hybrid  sigma  vertical  coordinate;  in  such  a  system,  the  lowest  levels  closely  follow  the  model 
terrain  whereas  the  upper  few  surfaces  are  virtually  constant-pressure  surfaces.  As  formulated, 
initial  and  forecast  values  of  temperature  and  moisture  adjacent  to  (2  m  above)  the  model  surface 
are  derived  from  data  at  the  lowest  model  sigma  level  by  assumption  of  a  well-mixed  (viz., 
constant  specific  humidity)  adiabatic  surface  layer. 

NORAPS 

The  Navy  Operational  Regional  Atmospheric  Prediction  System  (NORAPS)  consists  of 
a  multivariate  optimum  interpolation  analysis,  a  non-linear  vertical  mode  initialization,  and  a  21 
level  gridpoint  forecast  model.  The  NORAPS  analysis  method  is  adapted  from  the  NOGAPS 
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MVOI  scheme,  and  includes  a  12  hr  incremental  update  cycle.  Similar  to  NOGAPS,  the  regional 
model  does  not  have  a  moisture  analysis;  rather,  the  12  hr  forecast  field  from  the  previous  model 
run  serves  as  the  moisture  analysis  at  the  next  analysis  time.  During  the  forecast  initialization 
process,  meteorological  fields  are  interpolated,  assuming  a  linear  variation  with  the  log  of  the 
pressure,  from  the  analysis  levels  to  the  forecast  model’s  computational  levels.  Like  NOGAPS, 
the  regional  model  uses  a  first  order  K-theory  closure  formulation  for  the  planetary  boundary 
layer.  The  NORAPS  version  used  in  this  study,  which  provided  short-range  forecasts  out  to  24 
hr,  was  run  on  a  specially  implemented  40  km  horizontal  resolution  Lambert  conformal  grid 
covering  the  eastern  United  States  and  the  western  North  Atlantic  Ocean  (Figure  1). 
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